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ABSTRACT 



Repeatable accuracy of hydrographic positioning was 
examined in terms of the two-dimensional normal distribution 
function which results in an elliptical error figure. The 
error ellipse was discussed, and two methods for conversion 
of elliptical errors to circular errors were given. These 
methods are ’’circle of equivalent probability" and "root 
mean square error" (d rms )* Using the d rms error concept, 
repeatable accuracy of ranging, azimuthal, and hyperbolic 
systems was evaluated, and methods were developed to draw 
repeatability contours for those systems. 

A brief theoretical background was provided to explain 
the method of least squares and discuss its application to 
hydrographic survey positioning. For ranging, hyperbolic, 
azimuthal, sextant angle, and Global Positioning System the 
least squares observation equations were developed. Specific 
examples were constructed to demonstrate the capabilities 
of this data adjustment technique when applied to redundant 
position observations. 
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I . INTRODUCTION 



Positioning of the survey vessel is equal in importance 
with depth determination in the collection of hydrographic 
survey data. Fundamental to an understanding of the accuracy 
of position information is an analysis of the various errors 
and their sources which must be either eliminated, compen- 
sated for, or otherwise modeled. The result of this analysis 
is that the reliability of position data can be evaluated 
and used to estimate the overall accuracy of hydrographic 
soundings . 

Once these potential error sources are understood, 
methods must be developed to quantify accuracy. Much 
research has been conducted in this area in the past. One 
purpose of this thesis is to collect and present useful 
concepts of error theory which apply directly to hydrographic 
survey. Simple graphical techniques were developed which 
can be used to produce accuracy contours as a function of 
the survey net geometry. 

Conventional survey techniques rely on only two lines 
of position (LOP) to determine a positioning fix. This 
introduces the possibility of significant error. 

In navigation, although inherently less accurate than 
positioning due to the techniques and systems used to 
determine the LOP's, three LOP's are required to produce a 
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fix. Position is adjusted graphically by placing the fix 
in the center of the triangle formed by the three inter- 
secting LOP's. This concept of taking one redundant 
observation can lead to significant improvement in hydro- 
graphic survey positioning data. Mathematical adjustment 
techniques such as the method of least squares may be used 
to determine the best estimate of position. 

Least square adjustments are commonly performed on land 
survey data where redundant observations are easily made. 
With the advent of new positioning systems and computer 
technology, making redundant observations at sea is no 
longer impractical. The second purpose of this thesis 
is to explain the basic method of least squares, and to 
formulate examples of the least squares adjustment pro- 
cedure applied to specific types of hydrographic survey 
systems. This data adjustment technique not only provides 
the best estimate of position but also may be used to deter- 
mine the absolute positioning accuracy associated with each 
data point in a hydrographic survey. 
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II . REPEATABLE ACCURACY OF HYDROGRAPHIC 
SURVEY POSITIONS 



A. TYPES OF ERRORS 

It is impossible to make measurements of physical data 
without making errors. These measurement errors may be 
classified in the following manner. 

1 . Blunders 

These are mistakes which result from misreading 
instruments, transposing figures, faulty computations, etc. 

They may be large and easily observed, or smaller and less 
detectable, or very small and indistinguishable in the 
data. Blunders are usually detected through comparing repeated 
measurements, careful editing, and procedural checks in the 
data collection process. Physical measurements will contain 
a constant bias if these errors are not removed from the data 
set. 

2 . Systematic Errors 

Uncalibrated instruments or environmental factors, 
such as temperature and humidity changes which affect the 
performance of the measuring instruments, will induce system- 
atic errors into the observations. The occurrence of this 
type of error may result in a pattern which can be recognized 
and mathematically modeled. The simplest pattern to model 
would be some observable trend in the data of constant 
magnitude and direction. Such a trend can easily be 
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subtracted from the observations to remove the systematic 
error. 

If numerous systematic errors exist, or the errors 
are such that they cannot be accurately modeled, then their 
effect on the data must be estimated by calibration. Cali- 
bration is the process of comparing the measuring instrument 
against a known standard. The difference between the observed 
and known value may be used as an estimate of the total effect 
of all systematic errors present. Thus, calibration provides 
a "corrector" which must be applied to the data set. Examples 
of important systematic errors in hydrographic survey position- 
ing include instrument errors, errors in positioning control 
points, and variations in the propagation velocity of electro- 
magnetic energy. 

3 . Random Errors 

These errors result from accidental and unknown 
causes. Their effect cannot be removed from the observations 
and, therefore, must be quantified statistically. Random 
errors have certain characteristics which facilitate such an 
approach. Positive and negative errors occur with equal 
frequency, small errors are more probable than large errors, 
and extremely large errors rarely occur. 

The frequency distribution of random errors can be 
modeled mathematically by the normal distribution function. 
Assuming all measurement errors are independent and random. 
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thereby conforming to the normal distribution, measurement 
accuracy can be specified statistically by defining a con- 
fidence interval around the best estimate of the measured 
value. Procedures for computing these intervals are reviewed 
in Appendix A. 

B. .ACCURACY OF HYDROGRAPHIC POSITIONS 

The achievable accuracy of a hydrographic survey 
positioning system is best described by defining the follow- 
ing terms: repeatability and predictability. 

Repeatability is a measure of the accuracy with which 
the positioning system permits the user to return to a 
specific point on the surface of the earth defined in terms 
of the lines of position generated by the system. Included 
in repeatability are the effects of random errors, errors 
due to net geometry, and errors resulting from the angle 
of intersection for the two lines of position that establish 
a fix. Repeatable accuracy is therefore a measure of the 
relative accuracy of a positioning system. Unresolved biases 
exist in hydrographic positions due to the presence of 
systematic errors that have not been subtracted from the 
data or compensated for as a result of calibration. 

Predictability is the measure of accuracy with which the 
system can define the location of the same point in terms 
of geographic (or geodetic) coordinates rather than simply 
the intersection of two lines of position. Thus, predictable 
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accuracy is an absolute accuracy. Using conventional hydro- 
graphic survey techniques, predictability could be achieved 
only if all systematic errors were removed from the data so 
that only the effects of random errors, net geometry, and 
intersection angle remain. For example, the lattice generated 
by an electronic positioning system is distorted primarily as 
a result of the variability in the propagation velocity of 
electromagnetic energy. Ideally, if there was no distortion 
of the electronic lattice, then the accuracy of a position, 
corrected for any remaining systematic errors, could be 
quantified statistically in terms of predictable accuracy. 
However, since these distortions exist, the effective velocity 
of propagation would have to be accurately modeled through- 
out the survey area. Then it would be possible to subtract 
the effects of this systematic error and derive positions 
in terms of predictable geographic coordinates. Research 
is currently being conducted to quantify the parameters which 
affect propagation velocity in order to model these values 
for such application [Ref. 19], 

A second method to achieve predictable accuracy is by 
making redundant observations to establish hydrographic 
survey positions. If three intersecting lines of position 
are available instead of the usual two, the resulting fix 
is overdetermined, and data adjustment techniques must be 
applied. The method of least squares is most useful in 
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adjusting such data. Through the application of least 
squares adjustment techniques, the best estimate of position 
is found and the position's predictable accuracy is resolved. 

A complete discussion of this procedure is presented in 
Section III. 

C. REPEATABLE ACCURACY 

In the determination of hydrographic positions, blunders 
are eliminated by observing strict survey procedures, and 
system calibration is performed in an attempt to remove system- 
atic errors. Because some systematic errors still remain, 
the accuracy of hydrographic positions must be stated in 
terms of repeatability. 

The modeling of random errors is done by using the two- 
dimensional normal distribution function. When the normal 
distribution is applied to the positional errors, the result- 
ing error figure is an ellipse. 

1 . Elliptical Errors 

Hydrographic positions are determined by the inter- 
section of two lines of position (LOP). Because of the errors 
in each LOP, the actual position may lie somewhere between 
the error limits (shown as additional arcs either side of 
LOP’s in Figure II -1) . 

The intersection of the two LOP's, together with the 
standard errors associated with each, is drawn to an expanded 
scale in Figure II- 2. By applying the two-dimensional normal 
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Figure 1 1 - 2 : Expanded view of the intersection of 

two lines of position and the associated error ellipse. 



distribution to positional errors, it is seen that the contours 
of equal probability density about such an intersection are 
ellipses with their center at the intersection point. 

For simplicity in the discussion, the following assump- 
tions are made: 

1. Only errors contributing to repeatable accuracy are 
considered. 

2. The random errors associated with each LOP are 
assumed to be normally distributed. 

3. The random errors in each LOP are assumed to be 
independent, i.e., a change in the error of one 
LOP has no effect upon the other. 

4. The LOP's are assumed to be straight lines in the 
small area in the immediate vicinity of their 
intersection . 

5. Errors of position are limited to the two- 
dimensional case. 

As shown in Figure 1 1 - 2 , the general case of the inter- 
section of two LOP's at any angle and with different values 
of errors associated with each LOP results in an elliptical 
error figure. 

It is readily seen from Figure 1 1 - 2 that the exact shape 
of the error ellipse varies with the magnitudes of both of 
the one-dimensional LOP errors, and o^, as well as with 
the angle of intersection, g. 

The values of the semi -major and semi-minor axes of the 
error ellipse (using one o error) are given by the following 
equations [Refs. 4 and 5]. 
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Semi-major axis : 



= — f ( ff* + 6* ) + l/( <£)- (^S.'n a (3 ) <5? <s£ ' 1 • 

2Slo 2 £ L V -I * 



(Il-la) 



Semi-minor axis: 



<Sa«-± 



2 S;n 1 p 



+ 1 . cn-ib) 



Generally a i = CT 2 = a > then ec l ua tions (Il-la) and (II-lb) 
simplify to 



oy — 



flSx'n ^/2) 



and 



^ = 



^cos(P/a) . 



CH- 2 ) 



After computing the semi-major and semi-minor axes, the 
probability of the error ellipse is given by the distribution 
function 

Vl 

PU,y> = i - e 2 , cn-3) 



where 
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The solution of equation 1 1 - 3 with values of h for different 



probabilities yields the results shown in Table II-l. 



Probability (%) 


h 


39.35 


1.0000 


50.00 


1.1774 


63.21 


1.4142 


90.00 


2.1460 


99,00 


3.0349 


99.78 


3.5000 



Table 1 1 - 1 : Values of constant h. 



For example, for 39.35% probability the axes of the 

ellipse are 1.00 a and 1.00 a ; for 50% probability the axes 

x y 

are 1.1774 o and 1.1774 a . Figure 1 1 - 3 shows the error 
x y 

ellipses for different values of h. 

The angle, 0, between the semi-major axis of the error 
ellipse and the line of position which has smaller standard 
error is given by [Ref. 5] 



4-on 2 © 



g* Sin 1 ft 

ff, a Cos2 £ 1 



(II-4) 



h =4142 
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Figure 1 1 - 3 : Contours of equal probability areas. 



where is the smaller standard error. In the case of 
a = a^, equation II-4 simplifies to 

e =JL . (n-s) 

2 

The importance of the angle 9 is that it specifies the 
orientation of the error ellipse according to the lines of 
position. 

2 . Circular Error Approximations 

In general, the use of the error ellipse is compli- 
cated by the problem of axis orientation and the propagation 
of elliptical errors. Therefore, in order to simplify 
probability calculations and avoid the above problems, the 
elliptical errors are approximated by circular errors which 
are easier to use and understand. The accuracy of a hydro- 
graphic position may then be stated in terms of a circle of 
specified radius about the point. 

Note that when the angle of intersection is a right 
angle and the two errors are equal, the error ellipse becomes 
a circle and is described by the circular normal distribution. 
Generally, this is not the case, and elliptical errors must 
be converted to circular errors. This is done by using either 
the circle of equivalent probability or the root-mean-square 
error concept. 

a. Circle of Equivalent Probability 

A circle of equivalent probability is obtained 
utilizing an existing table for the two-dimensional normal 
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distribution (Table 1 1 - 2 ) . This table is used with the 
two standard errors along the semi-major and semi-minor axes 
of the error ellipse (Equations Il-la, Il-lb or 1 1 - 2 ) . To 
find the radius of equivalent probability, equations Il-la, 
Il-lb or 1 1 - 2 must first be utilized to obtain the values of 



o and cr , 
x y 



To enter the table the following ratios are needed: 



c = — where o is the greater standard error 

°x x 



and 



Y _ Radius of circle of equivalent probability 

Greater standard error > 

where K is the conversion factor needed to solve for the 
radius (R) of the circle of equivalent probability. 

The table relates varying values of ellipticity 
to the radius of circles of equivalent probability. Enter 
the table with the computed values for c and K to determine 
the probability for a circle of given radius, or alternately, 
for a given value of probability, determine the radius of the 
error circle. 

EXAMPLE 1 1 - 1 : The two standard errors of a positioning 

system estimated from field observations are - cr ^ = 6 meters. 

To determine the probability of location within a circle of 

10 m radius when the angle of intersection, 3, is 60°, 

equation II -2 must be used to find a and a : 

x y 
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7630305 


7532175 


. 7359558 


. 7070973 


6714209 


6306168 


5*03494 
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5132477 


. 8008554 


. 7929968 


. 7793550 


.7507265 ; 


■ 7249673 


6873122 


6474304 


6079*22 


5704420 


. 834(8)18 


8277048 


. 8169851 


. 7989288 | 


1 7720889 


i . 7383089 


| 7007900 


• <102303.* 


62168*9 


. 8627728 


. 8577362 


8493071 


. 8350816 


. 8129287 


1 7833962 


7480500 


7 1 22.14*» 


0753475 


. 88750(0) 


. 8834914 


. 8768644 


. 8657550 1 


! 8478303 


1 *220246 


7917194 


757470s 


t 7219027 


. 9085019 


. 9053700 


9001746 


. 8915536 


. 8773116 


8562471 


821)1137 


7077**2 


. 7462539 


9203125 


9237989 


9197275 


. 9130600 


9019110 


[ 8846624 


8613238 


*332175 


' *021013 


. 9411299 


939 1586 


9350865 


. 0308015 | 


. 9222277 


9083600 


*8*i*731 


*630149 


' . *355255 


9533775 


9518415 


. 9493815 


9454548 1 

j 
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9115702 


*901495 


8646647 
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! 8897495 


9715237 j 


[ . 9706109 


. 9691597 
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9631017 
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04.593*6 
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1 .9110784 


9780408 


9773450 


. 9762419 


9745239 


. 97111934 ! 


! 9067306 


| 9583739 


9458085 


. . 92*91)46 


! . 9832 1 HO , 
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. 9*05703 


. 9784061 


9747495 


1 0682698 


I 95*0*04 


1 . 943*652 


j 9872900 
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9862720 ! 


9853112 j 


9*37569 


9810035 


97<*)522 


9**79136 


. 9560031 


9904612 j 


. 9901674 


. 9897045 | 


. 9**0934 


0*7*-'. 27 


9*58331 


! . 0*21023 


' 9756069 


. 9659525 


. 9929062 ! 


! . 9920894 


. 9923483 i 


. 091*260 


. 9909944 


98052ft* 


, 9*07530 


, 9*17*37 


. 973*7*6 


9947727 
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.0943049 ; 


. 9930842 


9933*21 
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I . 9002*** 
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. 9*015*0 


9961 M3 4 
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.995*878 l 


. 996612ft 1 


995179* 


994424ft 


I . 99294*2 , 


| . 9900803 


. 9*50792 


. 9972391 j 


. 9971564 


. 9970264 , 


. 9968294 ! 


. 996.5205 


. 9959*54 


i . 994927) 


9927923 


. 0**8010 


. 9980212 ! 


. 9979622 


. 9978699 


. 997729ft 


9975109 


. 997134* j 


9063*51 


•404*16* 


. 90181 1:1 


. 9985949 


. 99H5533 


. 99*4880 


. 90*3*92 | 


. 99*2350 


. 9970733 


907147* 
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l . 9940240 


9990110 


. 9989824 


. 908936* 
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9985792 * 


1 9982 1 47 


9974O04 
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. 9993112 1 
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. 9992115 | 
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. 9990129 | 


| 99*7626 


09*1*6* 


i 9969113 


. 9995245 | 


. 9995105 


9994888 


. 9994550 ; 


9994053 


. 9993204 j 


; . 9091502 


09*74 sO 


9978125 


. 9996748 ! 


. 9906653 


. 9996505 


. 9996281 ! 


. 999* -938 j 


. 9995364 ! 


! . 999421* 


WO 1442 


. 90*4 6ft2 


. 9997797 


. 9097733 


.9997033 


. 90074*2 


99972.51 ! 


. 9996867 1 


. 999610*2 


. 999420* 


. 9989352 


. 9998522 


. 9998478 


. 9998412 | 


. 999*3 1 1 ; 


999*157 ! 


. 9997902 


. 9997396 


90061 10 


9002682 


. 9999018 


. 9998089 


.9998945 | 
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909*776 


. 999*60* 


. 990*276 
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. 9999261 
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9990085 1 
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1 . 9996045 


. 0999578 


. 999956ft 


. 9999547 


9999519 1 


. 4999475 j 


9999404 1 


9909266 i 


| 001)8900 1 


1 9907763 


. 9999727 


. 9999720 


. 9999707 


. 9999689 j 


. 99996ft 1 


9999616 


. 9099527 


9999292 ; 


. 090*523 


. 9999826 


. 9999821 


. 9999*13 j 


. 9990801 ! 


. 99997*3 


9999754 


. 090969* 


. 9099548 I 


t . 9009034 


. 9999889 


. 9999*86 


.9990881 t 


. 9999*74 


. 9999*63 


. 9990*45 ; 


; . 9009*00 


9009715 i 


, . 9999375 


. 9999931 


. 9999929 


.0999925 j 


. 9909921 


. 9999914 


. 9099902 


. 9009**1 


. 99 90 822 


. 9909599 


. 9999957 


. 0999955 


. 9999954 


9909951 j 


. 09914947 


. 9999939 


. 0900926 1 


. 90008*9 
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. 9099973 


. 0999971 


. 9999970 ! 


. 9999967 


9999063 


. 9099055 


9009932 


. 9909*40 


. 9990984 


. 9999983 


. 9990983 


. 99999*2 


99999*0 


9999977 


. 9009072 : 


. 0000950 


, 9009901 


9999990 


. 9999090 


. 9999990 | 


. OWOO^O : 


99999** 


. 90999*6 


. 90000*3 ; 


9000075 


. 9909939 


. 9999994 


.9999094 . 


. 9999994 | 


9999993 j 


9999993 


. 9999992 


9909090 


90000*5 


. 0000963 


. 9999997 


. 999’ >996 


. 9000006 1 


. 9099006 


4999990 


. 9000095 


. 0090094 ' 


01)0900 1 


. 999907* 


. 9999998 


. 0999998 


. 999999* 


. 9900908 , 


999**99* 


. 9999997 


9090997 


0000095 


. 99009*7 


. 9990999 


. 9999099 


. 9991>099 i 


01*91t09r. ' 


999091*9 j 
99*49999 * 


. 999999* 


. 9001)00* . 


9000007 


. 9909902 


. 9999999 


. 99909W 


. 9990099 


9990900 


. 9990099 


. 0990900 i 


. 90*1000* 


. 0999995 


1. OOOOOOO 


l. OOOOOOO 

j 


1. OOOOOOO 

i 


1 OOUUt »*) 


1 UOiHUKH) | 

| 

1 


9999999 
1. OOOOOOO 

1 


. 0000900 
1. OOOOOOO 


0099090 
. 0909909 

1. **8*0000 j 


. 9900997 

. 0999 90S 
9999999 
l. OOOOOOO 



Table X I - 2 : Circular error probabilities 

CBowditch, 1977). 
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CS\ 



- & — 8 - ^6 m 

fz 5in Wl 'Tx Si'o 6oy^ 



(5u = £ = —4.9m . 

^CosP/jZ \ftCcs£Q.° 

2 - 

Using the ratio, c. = -9^- = ttllL- — . 5 6> and 

<3* 8-46 

C — ro^>os of c-ioc-Le. _ 10 4s j. « 2. , 

<*V 8-48 

enter Table 1 1 - 2 with K 55 1.2 and c = .58 - .6. The proba- 
bility is found to be approximately 67%. (The value in the 
table is .6714269.) 

EXAMPLE 1 1 - 2 : For the system described in example II-l, 

the radius of the error circle with 90% probability may be 
determined. 

First, entering Table 1 1 - 2 with c = .6, for 90 

probability (the closest table value is .9019110), K is 

found to be 1.8. The radius of the error circle is equal 

to K times a : 
x 

1.8 x 8.48 = 15.3 meters. 
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Table 1 1 - 3 is more convenient for solving 
problems such as in example II-2 because the table is entered 
by using values of c and probability, P, in order to solve 
for the conversion factor, K. Note that the error circles 
identifying the 50% probability area (circular error 
probable, or CEP) and 90% area (circular map accuracy 
standard, or CMAS) are the most frequently used probability 
intervals . 

For constant values of a ^ and a circular error 

probabilities vary as a function of the angle of intersection, 

3, of the lines of position. To simplify the investigation of 

geometrical effects, the common case of = o ^ = o will be 

considered. Under this condition, the equations for o and o 

x y 

simplify to equation 1 1 - 2 . Taking the ratio of these two 
values, c is found to be C =. Gb / . Using 
the simplified equations, significant parameters of the error 
ellipse have been listed in Table 1 1 - 4 as a function of the 
intersection angle, 3, for the 50% probability interval (CEP) 
and in Table II -5, for the 90% probability interval. The 
data shows that the radius of the error circle, R, increases as 
the angle of intersection decreases. In the last columns, 
the error factor is defined as 



Error factor 



R (at any intersection angle) 
R" (at "3 = 90 u ) 
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or a multiplier by which the error circle radius, R, at any 
intersection angle may be computed from the radius of the error 
circle at 3 = 90°. For example, from Table II -4 it is seen 
that at a 50° intersection angle, R is 1.206 times greater 
than the radius at 6 = 90°. 

As seen in Tables II -4 and II -5, the optimum 
accuracy is obtained when the intersection angle, $, is 90°. 

It can be said that the geometric dilution of precision (GDOP) 
is minimum for a 90° intersection angle. Thus, the error 
factor defined in Tables II -4 and II -5 is commonly known 
as GDOP. Effects of geometric dilution are shown in Figure 
II -4 for CEP and 90% probability interval (CMAS) . Acceptable 
intersection angles for LOP's used in fixing hydrographic 
positions usually range between the limits of 30° and 150°. 

As seen in Figure II-4, the radius of the 90° probability 
interval circle is increased by a factor of two near the 
acceptable limits for hydrographic fix angles. Correspond- 
ingly, positioning accuracy is decreased by a factor of two. 

b. Root -Mean Square Error (d m ) 

^ v rms 

The root-mean-square error, d , is defined as 

rms 

the square root of the sum of the squares of the error 

components along the major and minor axes of the error ellipse. 

To calculate the d error, first equations II -la, lb or 

rms 

1 1 - 2 are utilized to obtain the values of a and o . Then 

x y 

the definition of d„ m „ is used, 

rms ’ 
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Figure II -4: Geometric dilution 

of precision for CEP and 90% probability 
interval (Bowditch, 1977). 



a rms = i 1+7F = 



where a * c is the semi -major axis of the error ellipse 

A 

and b = Oy. is the semi-minor axis. 

Alternately, formulas Il-la and Il-lb are sub- 
stituted into the definition of d error (equation II-6) 

rms 

and a more useful form of d^„ is obtained in terms of 

rms 

o 2 and the angle of intersection, 3: 



6 - i 

s 









Sin£ 



(II-7) 



Figure I I - 5 illustrates the definition of d error. 

rms 
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Figure 1 1 - 5 : Illustration of root mean square error. 



One d is defined as the radius of the error circle 
rms 

obtained using one cr x and one a y as the semi-major and semi- 
minor axes of the error ellipse. Two d rms is defined as the 
radius of the error circle obtained using two times the 

a and a values . 
x y 

The value of d rms does not correspond to a fixed 
probability interval for given values of and a 2 . It 
corresponds to a fixed probability interval only when 
3 = 90° and a. => a 7 so that the resulting probability figure 
is a circle. In the elliptical cases, the probability 
associated with a fixed value of d rms varies as a function of 
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the eccentricity of the error ellipse. This can easily be 
seen with an example using Table 1 1 - 2 . 

First, consider o x = 15 m, = 10 m, 

d rms = Vo s}%Oor = 18m , 

c = £k- = . 866 and K = ro<i:,ls ° f arror a.iS- =l.a . 

cy \S 

For c = .666 table values must be interpolated. Enter 
Table 1 1 - 2 with c = .6 and c = .7 for K = 1.2. The correspond- 
ing table values are found to be .6714269 and .6306168. 

Thus the probability of 18 m d is found to be 64.78%. 

' rms 

Secondly, consider a = 17 m, a = 6 m, 

x y 

<Vms =. V(\^ X + = 18 m , 

c - V|7- - - 353 onci \C - _ _Uo59 . 

If 

The interpolated probability from Table 1 1 - 2 is 67.4%. 

As seen above for the two cases, d errors are equal but 

c values (eccentricity) are different. As a result the 

corresponding probabilities are 64.78% and 67.4%. 

Table 1 1 - 6 shows the variations in probability 

associated with the values of 1 d„ and 2d as a 

rms rms 

function of eccentricity (a /a ), and Figure 1 1 - 6 shows the 

y 

same information graphically for 1 d error. 

r ' rms 
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Probability 



1 d 



rms 



2 d 



rms 



0 


.683 


.954 


.1 


.682 


.955 


.2 


.682 


.957 


.3 


.676 


.961 


.4 


.671 


.966 


.5 


.662 


.969 


.6 


.650 


.973 


.7 


.641 


.977 


.8 


.635 


.980 


.9 


.632 


.981 


1.0 


.632 


.982 



Table 1 1 - 6 : Variations in probability as a function of 

eccentricity (Bowditch, 1977). 




Figure 1 1 -6 : Variation in d rms with ellipticity (1 d rms ) 

(Bowditch, 1977) . 
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As seen in Table 1 1 - 6 , the probability that the 
position will be within the 1 d rms error circle ranges from 
68.3% when o = 0.0 to 63.2% when a = a and two d 

V y iv illw 

ranges from 95.4% to 98.2%, respectively. 

In Equation II-7, the d rms error was given 
assuming the errors in each line of position are independent. 
If the measurement of line of position #1 is related to 
measurement of line of position #2, then there is correla- 
tion between o^ and a e.g., is dependent on a 24 or a 
change in produces a corresponding change in a In 
this case, the equation for the root mean square position 
error is given as 



d rrnS = Vcr, 2 -* cl + 2p(T, ffaCosB , rTT 

Svn£ v l - ii ' 

where is the correlation coefficient between o^ and o^. 

Two different derivations of this equation are presented in 
the following papers: Bigelow (1963) and Heinzen (1977) 

[Refs . 2 and 10] . 

In summary, root mean, square error is easy to 
obtain mathematically, and it yields relative values of 
accuracy which are normally understood. Therefore, in subse- 
quent sections, d rms will be used to explain the repeatable 
accuracies of hydrographic positioning systems. 
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D. REPEATABLE ACCURACY OF HYDROGRAPHIC POSITIONING SYSTEMS 



1 . R anging Systems 

In ranging systems, the lines of positions are drawn 
as circles centered about each control station. The repeat- 
ability of this type of system is a function of the inter- 
section angle, B, and the random errors associated with each 
line of position. 

The two ranges are independent of each other. There- 
fore, the correlation coefficient, p, is zero, and d is 
given by Equation II -7 which is repeated here : 

d rcns ' . (III-9) 

5in p 

Usually, the standard errors of the two shore stations are 
equal. The system standard error, a s , of a time measuring 
positioning system is given as 

a l = a 2 = a s • 

The system standard error, o s , of a phase comparison 
positioning system is computed as a fraction of the lane width 
so that = Oj = aw = o s , where a is the standard error of 
range in fractions of a lane (i.e., a = .1 lanes) and w is 
lane width. Then Equation 1 1 - 9 reduces to 
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nJT 1 cr s 
S»r\ (B> 



J 



(I I - 10) 



d 



rms “* 



where a s is the system standard error. 

As seen from the above formula, the d „ is 

’ rms 

smallest at a 90° intersection angle and becomes large as 
$ approaches 0° or 180°. 

2 . Hyperbolic Systems 

As in ranging systems, the repeatable accuracy of 
hyperbolic systems is a function of intersection angle and 
random errors. Because landwidth is not constant for hyper- 
bolic systems, the change in lane width must also be quanti- 
fied. As the user moves away from the base line between the 
master and a slave unit, the lane becomes wider due to the 
divergence of the hyperbolic LOP’s [Ref. 8]. This divergence 
is expressed as an expansion factor, E ; 

E i = 1/sin (0^2) , 

where 0^ is the angle between the radius vectors from the 
position at p to the master and the respective slave station 
CFigure II-7). Then the standard error of one line of 
position at p is 



0"C =. cr sjv E i 







(a) 
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Figure II -7: A hyperbolic triad 



where a is standard error in the base line in fractions of 
a line, w is lane width and E expansion factor. 

The hyperbolic LOP’s bisect the angle between the 
radius vectors from p to master station and the respective 
slave station. Therefore, the angle of intersection, 8, is 

d _ 6> + ,9a (b) 

r 2 

Substituting equations (a) and (b) into II -8, d rms becomes 



^rms 



CvX/ 





Su?Ji 

X 



qp Ccs_(^J^A) 

SiO % • Svn ^2= 

2. x 



(11-11) 
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In a triad (three-station net) one range is common to both 
lines of position. Therefore, the correlation coefficient 
is not zero. Bigelow (1963) [Ref. 2] assumes a value for 
the correlation coefficient, g, of 0.33 while Swanson (1963) 
[Ref. 14] gets p = 0.4. Since the determination of this 
value is based on observations comprising a statistical 
sample, the most conservative value of d rms may be obtained 
by using p = 0.4. 

3. Azimuthal Systems 

In an azimuthal system, whether it is optical or 
electronic, the lines of position are radial vectors eminating 
from each of the shore stations. The repeatability of such 
systems is dependent upon the angular resolution of the 
system, and the angle of intersection of the radial vectors. 

The errors of position depend on: 

(1) the distance, r, along the radial, 

(2) the angular resolution, a, in degrees, 

(3) the angle of intersection, 3. 

The angular error may be expressed as an arc distance perpen- 
dicular to the respective radial at p as 

<rc = - S°- (a) 

SI- 23 b ' 

where r is the distance along the radial , a is angular resolu- 
tion, and 57.296 is conversion factor from degrees to radians. 
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Figure II-8: Azimuthal System Repeatability 



The two shore stations are independent; therefore, the corre- 
lation coefficient, £, is zero. 

Substituting Equation (a) into II-7, 



rrns “ 



51.236 Siofc 



. \f^l r T . 



(H-12) 



Applying the sine law to the triangle shown in Figure I I -8, 
it is seen that 



_ la 

©a S»o6» SvnC' 60 — (©»-»* 3 
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where Sin [180° - (9^ + 0 2 D 3 = Sin 8 = Sin ( 6 ^ + 62 ) and 
b = baseline distance. And Equation 11-12 may be written 
as 

dL mc =. ^ \j Sin a Q> -»• Sva^Qa* . 

5T. 196 Svn a (&»+© a ) 

Bigelow (1963) [Ref. 2 ] approximates Equation 11-13 by 
letting 

r» = r x s — _ 

2. S\n P/ 2 

Then Equation 11-13 reduces to 



Equation 11-14 is the approximate form of Equation 11-13. 
However, Equation 11-14 is easier to compute and the error 
introduced is negligible. For a = .03°, b = 8000 meters, 

= 80°, 0 2 = 30°, comparing the equations 11-13 and 11-14: 

using 11-13 



1 



(11-14) 



cons 



Sin p> • Svn fVl 



>( : rx5>«'l96 



^ _ («o3^ (Booo) 9 ^ 

rmS 57-196 * Sv^(&d+3d) 



\J Sm a 80°-+ Sxr^iO 0 



- 5.2m , 



using 11-14 




( - 03 D ( 8000) 1 

— — — - , | ** 



— 5.5m , 



\[X % 57- 296 Stv\ TO 0 Stc\ T0 c / a 
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it is seen that the difference between Equation 11-13 and 
Equation 11-14 is negligible. 

4 . Sextant Angle Positions (Three Point Fixes) 

The evaluation of repeatability for sextant angle 
positions is difficult. The mathematics involved in the 
computation are quite complex. Thus, repeatability of 
sextant angle positions is more easily evaluated by a graphi- 
cal analysis. For the development of an analytical solution, 
see Heinzen (1977) [Ref. 10]. 

As will be seen in later sections, it is much easier 
to derive the accuracy of sextant angle positions by applying 
the method of least squares. 



E. REPEATABILITY CONTOURS 

Using the root mean square error concept, one can con- 
struct a family of curves to display convenient values of 
d r m S terms of the system geometry. 

1 . Ranging Systems 

For ranging systems, the d rms is given by 
Equation II -10 as 



^rms 



<t& 



(II -10) 



. Note that the intersection angle, 3, is the only 

controlling geometric factor of d rms . Figure II -9 shows an 
example of suitable geometry for a ranging system. Mathemati- 
cally, it can be proven that the intersection angle, 3, 
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is equal to the angle formed by radius vectors from p to 
the slaves and also the angles S^OD and S 2 OD. The locus of 
points having a constant d rmg and constant 3 describes a 
circle of radius r = b/2 Sin 3 with the two shore stations 
as points on the circle. 

The distance, e, along the perpendicular bisector of 
the line connecting two shore stations to the center of the 
circle is 



e - k 

0. ton ^ 



(II-15) 



1_ / o 

(since, from Figure II -9, tan 3 = -^-=0 where b is the distance 

V * 

between shore stations and S^. 

Using 2 o g error (approximately 951 probability 
interval) , Equation 11-10 may be written in the following 
form 

^rtr>s _ 'Z 2.' 

«s Sfn£ 



Writing Equation 11-15 as 

<2. _ 1 
2-tao 

Figure I I -10 was constructed to show the relationship of 
d r ms/ a s anc * e /k as a function of intersection angle, 3. 
Using this graph, selected contours of constant d rms may 
be drawn as in Figure II -11. First, plot the location 
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Figure II -9: Ranging system geometry. 

of the two shore stations at a convenient scale. Draw a 
perpendicular bisector to the line joining them. Using 
Figure II -10 determine the values of e/b for the desired 
d „ contours. From the known value of b, determine distance 
e for each contour. Lay off distance e along the perpen- 
dicular bisector to define the center, 0, of the desired 

constant d circle. The radius of the selected contour 

is the distance from the center, point 0, to the shore 
stations . 
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Values of d 



Example I I - 3 : A phase comparison range-range 

positioning system has standard error a g 3 .01 w (lane width). 
It operates at 2 Mhz frequency. The distance, b, between two 
shore stations is 20,000 m. 



Lane width , w = 



300,000 _ 

2 x 2000 " 



7S i , 



a » ow 3 75 x .01 3 .75 m . 
s 



For the 2 m d rms contour, d rms / a s 3 2/. 7 5 3 2.66. 



Enter Figure II -10 with d rms / a s 3 2.66 which intersect the 




10° 20° 30° 40°50®60®70®80 o 90° 

170° 160® 150°140 o 130 o 120 o 110 o 100° 

Intersection angle, 8, in degrees 



Figure I I - 1 0 : For ranging systems, the graph of the 

d rms / a s an< * e /b* enlarged figure, 

see Appendix B.) 
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the e/b curve. It intersects at e/b = .043. For this 
specific pair where b = 20,000 m, 

e = b ( • 043) = 860 m 

Using the described technique, the 2m d rms contour 

can be drawn, and the result is shown in Figure 11-11. 

Thus between the 2m d contour for 6 = 85° and the 2m d^, _ 

rms . rms 

contour for 8 = 95°, the d„ m error for the described system 
will be 2m. 951 of the time. 

Note that when the angle of intersection, 8 = 90°, 
d rm s error minimum. Therefore, as B increases toward 
180° or decreases toward 0°, d becomes larger. Because 
the tangent of the angles greater than 90° is negative, 
e values will be negative as well. Thus, the center of 
the constant d rms circles, for angles greater than 90°, 
will be on opposite side of the baseline. As shown in 
Figure II -11, d rms error increases as the baseline is approached. 
Contours for d values of 3, 4 and 5 meters may be constructed 
by following the procedures outlined above. 

2 . Azimuthal Systems 

For azimuthal systems, d error is given by 

7 rms ' 

Equation 11-14 as 

ci _ Q. t ^ 

°rms - — — ' — TTT~ j 

'JT *5f*296 S.’n ?/% 
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Figure I I - 11 : Repeatability contours of a ranging pair. 



where a is the angular resolution, measured in degrees, 
b is the distance between two azimuth stations and 3 is 
the intersection angle of radial vectors which is defined by 
the equation 3 = 180° - ( 0 ^ + 0 2 ) (Figure II- 8 ). As with 
ranging systems, the intersection angle, 3 , is the only 
geometric factor contributing to d rms . Constant error con- 
tours are obtained in a similar fashion. 

Writing the Equation 11-14 with 2 a error (approximately 
951 probability interval) as 

drms _ 2 1 

( 57 . 296 )* Smp.Svo P/I ' 

where a = angular resolution and b = baseline distance. 

Since e/b = 1/(2 tan 3), a graph is drawn showing d rms / a * b 

and e/b as a function of intersection angle, 3 (Figure 11-12). 

Figure 11-12 provides a convenient means to obtain 

the values of e distance for a selected d if a and b 

rms 

are known. 

Example I I -4: The distance, b, between two azimuth 

shore stations is 2,000 m. Angular resolution of the station 

is a 1 = a 2 = . 01 °. 

For the 2 meters d„ contour 

rms 

^rms = 2 

~a T5 - " .01 x 2000 * 1 * 
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10 20 30 40 50 100 200 



Values of angle 0 (in degrees) 

Figure 11-12: For azimuthal systems, the graph of and _i 

a* b 

CFor enlarged figure, see Appendix B.) # 



Enter Figure 11-12 with d „/a*b = .1 which intersects 
& rms 

the d / a-b curve at 44°. Follow 44° line vertically to the 

rms ' 

e/b curve, which intersects at e/b = .52. For this pair where 
b = 2,000 m, e = 2,000 x .52 = 1040 m. Using the technique 
as described for ranging systems, the 2 m d rms error contour 
may be drawn (Figure 11-13). Other contours are computed 
in same manner. 

Note that when 3 > 90°, the tangent value is negative 

and the center of constant d „ circle will be on the 

rms 

opposite side of the baseline. For azimuthal systems, the 

minimum d„„„ error is found at 3 = 109°. 
rms 

3 . Hyperbolic Systems 

The root mean square error for hyperbolic systems 
is given by Equation 11-11 as 



Tms — 




Sfc\ a ®> 

UL 



2 e cos g 



Svnf.Stn^ 



( 11 - 11 ) 



where a is the standard error along the baseline between the 
master and respective slave station in fractions of a lane, 
w is the lane width and 3 is the intersection angle which is 
equal to 

e + 0 

3 = — L— — £ (Figure II- 7 ) . 
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Figure 11-13: Repeatability contours of an azimuthal system. 
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Equation 1 1 - 1 1 is written with 2 o error as 



2= V / t | -l + 2 (,^CosP 

VS’vo' 1 - £» Svn a &?- Sin ©1 - Svc\ ii 

a 2. a 2 . 

where the correlation coefficient, ^>, was taken as .4 
[Ref. 14], Figure 11-14 was produced to show the d rms /aw 
values as a function of the angle subtended by the two slave 
stations , i .e . , 26 . 

A parameter, p, defined as 

p _ 6,1-6! _ er p _ 6,1-6! _ 2 JL 

©I ©| ©2 ©2. 

The parameter, p, is computed with the smaller of the two 

angles, or , in the denominator. Thus when p = 2 

the master station is positioned on the bisector of the angle 

subtended by two slave stations, p = 3 places master station 

on one of the two trisectors, and so on (Table II-7). 

Knowing the angle subtended by the two slave stations 

at a particular point, Figure 11-14 may be used to develop 

contours of constant d „ . First determine the d /aw ratio 

rms rms 

for a selected d rms • Enter Figure 11-14, for several values 
of parameter p, and read the corresponding values of angle 26. 
Using the relation between 26 and 0-^ ( 0 2 ) (Table II-7), plot 
these angles, 26 and 0^ ( 0 2 ) > on a conveniently scaled chart 



drms 
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Values of d 




2$, in degrees 

Figure 11-14: In hyperbolic systems, for several values of p, 

d /crw curves, 
rms 
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p 


6 1 or 6 2 


2 


(28)/ 2 


3 


(28)/3 


4 


(28)/4 


5 


(28) / 5 


6 


(23)/ 6 


7 


( 2 8 ) / 7 


8 


( 2 3 ) / 8 


9 


( 2 8 ) / 9 


0 


(28) /10 



Table 1 1 - 7 : Relation between 28 and 9^ or 
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with a three arm protractor. Interpolating between the points, 

draw the d contour. The curve thus determined defines the 
rms 

location of a selected d contour for the specific conditions 

rms 

of triad configuration. 

Example 1 1 - 5 : A hyperbolic system has standard 

error, a, equal to .01 lanes along the base line. It 
operates at a frequency of 2 Mhz. Triad configuration is 
as seen in Figure 11-15 : 



lane width. 



„ . v _ 300,000 
2T 2 x 2,000 



75 m 



aw = 75 in x .01 = .75m, 



For the 4 m d _ contoured /aw = 4/. 75 = 5.32. Enter 
rms rms 

Figure 11-14 with d rms /aw = 5.32. For several values of p, 
read the corresponding values of angle 20. Determine the 
values of angles 0^ or 6^ according to Table 1 1 - 7 . For the 
4m contour, these values are shown in Table I I- 8. Using a 
three arm protractor, the points defining the 4m d rms contour 
may be plotted. The other contours are drawn in a similar 
manner (Figure 11-15). 
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eS 




Figure 11-15: Repeatability contours of a hyperbolic system 

(a = .01 lane width and f = 2 mhz) 
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Ill . APPLICATION OF LEAST SQUARES TO 



HYDROGRAPHIC SURVEY POSITIONS 

A. THE PRINCIPLE OF LEAST SQUARES 

Given a set of unknown parameters to be computed from 
measured physical quantities such as distance or azimuth, 
the least squares method provides a mathematical procedure 
by which the best values for the unknown parameters may be 
obtained. 

Equations must be written to define the relationship 
between the observed and the unknown parameters. If the 
number of equations that can be written is equal to the number 
of unknowns, then a unique solution may be computed. 

However, no statement can be made about the accuracy of 
the solution. In the least square method, the number of 
equations must be greater than the number of unknowns. 

As a result of this over-determined solution, the best values 
for the unknown parameters are estimated. 

This computational procedure is referred to as a least 
squares adjustment. In application, corrections are com- 
puted and applied to observed quantities and these quantities 
are then said to be adjusted. 

For a given set of equations, the fundamental condition 
of the least square technique requires that the sum of the 
squares of the residuals be minimized. A residual is defined 
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as the difference between an oberved value of a quantity 
and the arithmetical mean value of that quantity obtained 
from a number of observations. If the arithmetical mean value 
is stated by and observed value by x^, the residual, v, 
is expressed as 

v - x. - x. . (III-l) 

Suppose a set of observations were taken having residuals 

v l , v v^ v . Then in equation form, the fundamental 

condition of least squares is expressed as 

^ — {\f\) + {Hi) + , . - » 4. — mvmonum. (II I- 2) 

T 

or in matrix form: V V = minimum. 

1 . W eighted Observations 

In general, some of the observed values may be more 
precise, and, therefore, entitled to have greater influence 
upon the result. Observations are assigned values called 
weights corresponding to their quality or worth. 

The assignment of weights to observed values is 
largely a matter of judgment. For example, if one set of 
measurements of a distance was made with four repetitions 
and another was made with eight repetitions, the mean of the 
second set of observations may be given twice the weight 
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of the first set. Or, when measuring angles in azimuth angle 
positions, the atmosphere may be so unsteady during one obser- 
vation that the observer arbitrarily assigns a weight of 
one half. 

As a general rule, if a standard error, a, has been 
computed for a set of observations, then weights are usually 
estimated according to the equation 

•=. JiL , (III-3) 

where w^ is the weight of the ith observed quantity, 
is the standard error of that observation , and k is any 
number which has the same value for all observations. 

Equation 1 1 1 - 3 states that weights are inversely proportional 
to the square of the standard error. Usually, the weight 
corresponding to the least accurate measurement is assigned 
a value of 1 (a unit weight) . Then the value of k can be 
found and the other weights computed accordingly. 

For example, consider the standard error for two 
observations where = 3, and a 2 = 1.5. Assigning w^ = 1 t 




Wl = - 9 — 4 

*1 ( 1 . 5 ^ * 
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it is found that the second observation has a weight of 4 relative 
to the first observation. 

If measured values are to be weighted and used in a 
least squares adjustment, then the condition is that the sum 
of the weight times their corresponding squared residuals 
must be minimized, 

_rv 

X + + _ - m ^' vraurn » (1 1 1 - 4) 

or in the matrix form, V := mioimum . 

2 . Method of Least Squares Adjustment 

In the "observation equations” method of least 
squares adjustment, the observed quantities are related to 
the desired unknown quantities through formulas or functions 
which are called observation equations. 

One observation equation is written for each measure- 
ment, and it is assumed that observations are independent 
of each other. In order to solve for the best value of each 
unknown parameter, at least one redundant observation equation 
must be written. That is, the number of observations must 
be greater than the unknowns. 

Observation equations may be linear or higher order 
functions. Linear observation equations can be written in 
general as follows : 
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a,* +b,y + c v t - -t- = <3. 

+ b a y + C2 *e + . * . . . * *■ ka 

■ 

■ • « * I 

, \ < ‘ * 

* i * • 

; » * ! ; 

a 0 l + \o n y + c n -k i- -»- k n - G n ^ 



(1 1 1 - 5 ) 



where a's, b’s, c's, etc. are coefficients of unknowns x, y, 

z, etc. and the k's are constants. 

Because the observations (G. , G,. G ) are not 

12 n 

free from random errors, each G^ must be corrected by a 
residual value, v^, in order to obtain a mathematically 
correct equation system. Thus, 



o,* -*• +• c,l + + 

Oa* •+■ -+ c.-l 1 -*■ + 

- ' * , 

\ • 

' ! : I 

* , , i 

: i * , 

Qo K + V> n y ■+■ C n t + — y ^ 



= -f^i 
= Gi -*-Vi 

i 

i 

\ » 

i * 

- . 



CIII- 6 ) 



Introducing a new notation = G^ - k^ , = G^ - k 

etc., the following equation is obtained; 
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o i * + V>ig +c,if .. 



02 * + b^y + Cxi + 

1 , « 

' • i 



O n * + 



9 



+ c n ^ 



--i, =V, 

- - — ^2. = ^2. 

I 




(III- 7) 



or in the matrix form, V = AX - L 



(1 1 1 - 8 ) 



This equation is called the observation equation or observa- 
tion equation matrix, where 



’ V, ' 




Qi j V>\ , C.| (T> i 




A - 


Ql > ba , c x m2. 

• 1 1 t 


: 


n m 


i ! ; i 


v °. 




: ] : : 

, b n , Cn m n 
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In the above matrices, the subscript n denotes the number of 
observations and m denotes the number of unknowns. 

For a group of equally weighted observations, recall 
that the following condition must be enforced in order to 
perform a least square adjustment: 

n 1 

= minimum , 

Ul 

or in the matrix form, 

T - - 

V V = minimum . 

Substituting the value for the V matrix from the observation 
Equation III -8 where V = AX - L , 

V T v = cax-l) t (Ax-l) 

= (A%-0 C (a-)O t = A t A t ] 

= ( A T - iJ” ) l AX — (-from ma-tcvx. algebra) 

= K t A t AX - A t A t L - L T AX + C L_ 

and from matrix algebra, 1_7 AA -= K T A T ^- ■> 
then 

V T V = X t A t AX - 1 x t a t l +l t l . 
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The minimum of this function can be found by taking 
the partial derivatives of the function with respect to each 
unknown or with respect to the X matrix (which contains all 
of the unknowns) and equating it to 2 ero, i.e.: 

— ( V T V } = 2 A. T A X - 1 K T u = 0 . 
aX 

Dividing by 2, the following result is obtained: 

a t ax - a t l =o. (m-9 ) 



This is called the normal equation, 
the normal equation (III-9) becomes 


In conventional notation, 


too] * + Cably + Cacli ^ _ 


-Caf] =0 


[V>al x CC bl y + CV}cl"£ + . 


-tbf] =0 


Tea] X + Cc.t»]y + Cecil v 

' ! ' 

! : 1 


-CcU -0 

1 


l i ; 

[nal* + tnU3y + Cnelt + _ 


T 

-toil -0 , 


where the symbol [ ] denotes the sum of the products, i.e., 


[aa] = a^ ♦ a 2 a 2 - * 

+ b 2 a 2 + b e a 3 + * W 


W [bal = b l a l 
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T 

In Equation III-9, A A is the matrix of normal 

equation coefficients of the unknowns. Multiplying Equation 
T - 1 

1 1 1 - 9 by (A A) and reducing, the solution is obtained. 



(A T Af‘(A T MX. - (A T A) i A T L •= 0 , 

X = (A T AfV\- . CIII-10) 



Equation III-10 is the basic least squares matrix equation 
for equally weighted observations. The matrix X consist of 
best values for the unknowns x, y, z, etc. 

For a system of weighted observations the funda- 
mental condition is 



J <2. 

^ i — mini morn , 

L=i 



or in the matrix form 3 V T SAV = mini on urn . 

The normal equation matrix is derived similarly to 
the unweighted case. 



A t N fj AX - A t \*/L = 0, 



(III-ll) 



or in conventional notation 3 



67 



C \Naa 3 * 






-+• 

L ^ W a 3 \ C vj 3 

f 1 

» I 



Unolx + C.\A/nb3 



- Lwa/] — o 

-Cwbf] =0 

I 

I 

I 

-C wnf] - o . 



In Equation III-ll the matrices are identical to those of 
the equally weighted equation, with the addition of the 
matrix , V/> which is a diagonal nxn matrix. 

In detail, VS/ becomes 



vw* o 



0 



0 



\N = 



o 

0 

0 



\N Z 0 0 

0 \A/ 3 0 

0 0 



where according to Equation III -3, 



(III-12) 




The best values of unknowns are obtained by solving 
Equation III-ll as 
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* = ( A t \^aV 1 A t nW\_. 



(III-13) 



From the combination of Equations III-8 and 1 1 1 - 9 or III-8 
and III-ll, it is seen that 

A T (VtO ~A T L -O or 

K \N (V-tL") - K\N L - o. 

Therefore, 

A T V ^0 or A t \/ 7 V = O. (III- 14 ) 



Equation III-14 can be used as a check on the computation. 

Example III-l 1 : As an elementary example illustrating 

the method of least squares adjustment by the observation 
equation method, consider the following equally weighted 
observations : 

a*, + 3*x + *3 = io 

— 2* a + 3*3 —5 
"Hi + -'2*3 = 3, 

-%i - - *5 = -6 . 



1 The numerical values of this example problem were taken 
from Ref. 17, page 517. 
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These four equations relate the three unknowns x^, x 2 and 
x 3 to the observations. 

By including residuals, the equations may be 
rewritten as observation equations as follows : 

2 = \0 ■+ Vi 

\\ _ 1^2 

1 _ 2*3 = 3 + ^3 

- *i - >>3 =-t+V4 } 

or in matrix form. 



where 






~ 4^3 3^i ■» 



A - 



13 1 

1 -2 3 

2 1-1 
i -1 -1 





*1 




10 




v» 














II 


*1 


J L - 


5 


j V = 










3 


✓ 


^3 




u 










- 






- 6 




^4 
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The normal equation is 



A T A * - A T L - O ; 



A T A - 



1 I 1 -1 

3-2 1 -1 

1 3-1-1 



1 

1 

7 

-i 



3 1 

-2 3 

i -2 

-i -i 



A t L = 









- 


- 






- 


p 










— 






2 


i 


7 


_1 




10 




52 


3 


-2 


i 






s 

3 




28 


1 


3 


-2 


-i 




-£> 




25 


*■ 










H. - 




L 



55 


12 


-8 


12 


15 


"A 


-8 




15 





55 


12 


-8 




%» 




52 


A T A a _ A t l =. 


12 


15 






Ka 


— * 


29 




-5 


-4 


15 

_ 




*3 


■ 


25 

Ur J 



And the solution is X = (A T A) A^ L_ j 



(aVT** 



.022959 
- .016181 
. 007815 



- .oitiei 
. o 8 3233 
* 013 5623 



- 00 787A8 
.0135623 
. 014^83 
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. 02 2 853 
-.Oi8»8? 

. 00 ?875 



- . 01613 ? 
,082233 
.OI3S823 



, OOlST-hK 
. 0«3 58X3 
- 07 4 8 £3 



£>1 

29 

25 



. 9\ f>l 



X = 



1-8U09 

2.K488 



Thus the best values for the unknown parameters x^ , x 2 and 

x^ are x^ = .9161, x 2 = 1.91109 and x^ = 2.66488. 

This computation was performed by requiring that 
T 

V V = minimum. Thus, when the best values are used in the 
equation V = AX - L, the resulting minimized residuals can 
be found. If the minimized residuals are applied to the 
observations then the observations are said to be adjusted. 





X 3 1 




-SKI 




10 




> 23035 


V ^ AA-L = 


1 -2 3 

? i -1 




1.91109 


— 


S 

3 


— 


* 0885 6 
--053T 




-1 -i 4 




2.6849 




-6 




-50T93 
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23035 



V = 



. 08 BS6 
-.0597 



. 50793 

Then adjusted observations are G^ = 10.23035, G 0 = 5.08856 



G 3 = 2.9403 and G 4 = -5.49207. 

T 

Computational check: A V must be equal to zero 

according to Equation III-14. 





2 1 ? -i 




. 23 035 


A r v ^ 






. 08856 


3 -2 i .1 












— OS9T 




1 3 -1 -L 




. S0?9i 




p 




L 



- o oo 
. o o o 
.QOO 



According to the theory of probability, the 
above values of x^ , y <2 and x^ have the highest probability 
of occurrence. 

Example II I- 2: Suppose the constant terms 10, 5, 

3, and -6 of the observation equations of Example III-l 
represent measurements having relative weights of 1, 2, 2, 
and 3, respectively. Using weighted least squares, best 
values for x^ , and x^ will be calculated. 
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The observation equations in Example III-l were 



2 ■+ 3xi T X 3 = \0-tV t 

Xi - 1 *2 + 3X3 = 5 + 

7*, -f -*2 - IX 3 - 3> -f ^2. 

- *i _ %2. - x, 3 * -fc TVif ^ 



or in the matrix £orm > 



V = AX - L , 



where 



N = 



2 2 i 

i -2 3 

T i “2 

-i -i -I 






x, 

Xj. 

X 3 



, *-« 



10 

5 

3 

-6 



V = 



v, 

Vz 

*3 



The normal equation for weighted observations is 

A T WAX - AaA/ L =0, 



where weight matrix W is a diagonal matrix of weights as 
follows: r -» 



\*y = 



i 0 0 o 
0 2 0 0 
0 0 2 0 
0 0 0 3 
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A"V/A- 



A T VvJL - 



3 -2 i -i 
1 3 -2 -1 



a i f -i 
3 -2 i -i 
1 3-2-1 



n 


- -i 


— 


* 


1 0 0 0 






0 2 0 0 






o o 2 o 






0 0 0 3 





a 3 
1 ^2 
1 



10 0 0 
0 2 c o 

c o 1 o 
o o o 3 



“ 








1 




lOT 


19 


3 




19 


22 


-2 








-1 




-1? 


-io 


- 


10 




3o 




5 




34 




3 








-6 




— 1 
-r 



anti 

A T nA/ A* - A T \AL 



10? 19 -17 

13 22 -10 

-a -to 30 





X. 




» «■ 

30 




Xa 


- 


34 




*3 




^6 



-O 



The solution is V, = (A T vVA ) ' A\VL ; 



* - 



X = 



OH 38 33 --OOSI3I4 , 0 0 37^ 

..oo3\it4 ,0593795 , 015185 

■ 00 374 » 015 13 5 4 .04 0514? 



. 3201 
1.9856 
2,7-166 



90 

34 

46 



or X* = .92ol , X a = 1.9856 and * 3 = 2.7166 
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Residuals are found using Equation 1 1 1 - 8 : 



V=AX-L= 



r— — 

2 3 1 








lo 




. 5136 






-9201 










1-13 




L.9356 




5 


Z=L 


» c>9 &J 


>1-2 




2.J\U 




3 




-.oo69 


-1 -1 -1 








-6 




. 3m 


check: 




A T WV 






must be 



zero. 



a t wv 



1 

H 

)r~ 

t 

t i 




1 0 0 0 


3-11 -1 




0 2 0 0 
0 0 2 0 


i 3-2-1 




O 

a 

o 

U) 






* 



* 

. 5136 




.005 


.090? 






-.00 69 




.000 


. 3 >m 




. ooo 



3 . Higher Order Functions 

The observation equations presented by Equation 1 1 1 - 8 
are linear equations. If this relationship is nonlinear, 
thus defined by a higher order function, then the observation 
equations must be linearized in order to apply the least 
square adjustment method. 

Defining the general observation equation as 
G = f(x, y) , where f represents a non-linear function. 

The function must be linearized by Taylor series expansion 
or by some other method. The best values of x and y can be 
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regarded as the sum of an approximate value x q , y Q and a 
small correction Ax, Ay. Therefore x = x Q + Ax and 
y = y Q + Ay and the above function is written in. the following 
form 



G - f(x Q + Ax, y Q + Ay). 

Using Taylor series expansion the observation equations may 
be linearized. 



£ = -t- A* 



A3 Wigher order 4erms 



Cl I I - 14) 



The higher order terms in the series are neglected and only 
the zero and first order terms are maintained. 

After linearization, the observation equations 



become 



_ ~ S*'iU»9) £ 7>f,U ->y ) 



^ y. -+- - ” J — Ay + So') - Si 

- 2 >*o 



v a = 



Dy.o 



A 3 4- 

■sy*. 



— Sr 



CIII-15) 



'n 



t 

_ A. 



**+ - * J ^ A 3 &n , 

^*0 ~ay 0 
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or in the matrix form, V = AX - L, where 



~b£. 



T>yo 




i 

i 

i 

“Mn 

o 






“oyc 



- 




N/ = 



L = 



Qa-f 2 ( 

, 4 

I * 

6i\ -? n 



V, 

Vi 

I 



i 

i 




The remainder of the least square procedure is the same as 
indicated by Equations III-9, III-10 or III-ll, III-13. 

In the linearization process, the higher order terms 
were neglected. For this assumption to be valid, Ax and 
Ay should be small so that their products in the series 
expansion approach zero. (Ax • Ay = 0). This can be achieved 
only if the values of x Q and y Q are very close to the values 
of x and y. Therefore, x Q and y Q must be precomputed, or 
the original assumed x Q and y Q must be improved by successive 
iterations until the adjusted observations equal the measured 
values . 
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4 . Equations for the Precision of Adjusted Quantities 
After calculating the best values of the unknowns, 
or X matrix, the V matrix, or the adjustments to the observations, 
can be computed from the observation equation which is 
V = AX - L, whether the observations are weighted or not. 



Using the V matrix, the standard error of an obser- 



vation of unit weight is given by the following equations 
[Ref. 17] : 



for unweighted observations, 




(III - 16a) 



for weighted observations 




cm - i6b) 



where 



o Q is the standard error of an observation which has unit 
weight , 



n is the number of observations , 
m is the number of unknowns . 



79 



Standard errors of the best values for the unknowns are 
then given by the following equation: 



OC 




(III-17) 



where 

<j^ is the standard error of the ith adjusted quantity, 

e.g., the quantity in the ith row of the X matrix, 

a is the standard error of unit weight as found by 
o 

Equation III-16a or III-16b, 

T - 1 

q^ is an element of, for unweighted case, (A A) 

T - 1 

or, for weighted case, (A WA) matrix. 

T -1 T -1 

If the (A A) or (A WA) matrices are written in detailed 
form as 

(A T Af ortA’V'/Af = 



then the standard errors of the best values of the individual 
adjusted quantities are: 

0*1 •=. Co n! cjn * , 



— 

Sat R23 — 

^ 31 %2. S X - - - 
! 1 ' 
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CTx — cr 0 \j Cj ai ' ^ 
cr 3 - cr 0 \fc\^P ^ 

Example 1 1 1 - 3 : The standard errors of the best 

values for x^ , x^ and x^ in Example III-l. 

Standard error of unit weight for unweighted 
observation is 



CT 0 = 



r VTV_ 

n-m 



V T V 



23035 .08956 -.00537 .50793 

\ 



V T V- .313 , 



CT 0 





319 

4-3 



56S . 



2303S 1 
08956 
.00597 
507-93 
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The standard errors of the best values are given by 



Equation 1 1 1 - 1 7 as cr^ — • 

For unweighted observations, q^’s are the elements 
T - 1 

of CA A) which was calculated in Example III-l. 



(A T A^' 



.0 2X859 
--OlfclSf 
. 0O18T-5 



.D0?g>5 

. 083233 . 0l35£>23 

, 013.5623 . 07 - 44-83 



= cr 0 \T^T = - 565 >/TwIiir -=■ « 085 , 

<7*^ = cr 0 \[ 92 a • 565 \/. 0S32/33 

cr* s = o-o s fq^ - . S6S n/.OT^BV - -I&4-. 



T -1 

In the (A A) matrix, off diagonal terms are used 
to find the covariances of unknowns. Covariance 
is equal to 

0"n_ — cr c ^ 

From covariances, the correlation coefficients of variables 
are obtained. Correlation coefficient £ 12 is given as 



e 



12 



cr,i 



<7, 0*2. ' 
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The interpretation of the standard errors computed 
above is that there is a 68% probability that the adjusted 
values for x^ , and x^ are within ±.085, ±.163 and ±.154 
of their true values, respectively. 

Example 1 1 1 - 4 : The standard errors of x^ , x^ and 

x 3 in Example III- 2. 

The standard error of unit weight for weighted 
observations is 



da =\ N T ^ V 
n-rn 



- 



5136 .0 9 81- -.0 0 69 . 371 - 1 - 



1 0 0 o 




.5136 


o i o o 




J 09S> 


0 0 2 0 




-.00 69 


0 0 0 3 




.3171 


• 




■ -■ 



\ 



\J T V/V = . 1-114 , 



CT 0 — 



V T \MV 
n-m 




843 . 
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The standard errors of best values are given as 



<n = Co \/q a ' . 

T - 1 

For weighted observations, q^’s are the elements of (A WA) 
which has been already calculated in Example III-2. 



(A t nNA^ = 



* Oil 38 39 
oo 8V31H 

. oo 374 



-.00 81 31 4 

. 05 9 37-35 
. 0151854 



„ 00374 
, 0151859 
. 090514 



G\ = (To \/, 0113839 1 _ ,0S99 , 

«r^=a- e ^r =.8 H^-o59«s,.2os^ 

<r„ « Co {qTT = • 8W \|* oitosi^' = - im. 
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B. APPLICATION OF LEAST SQUARES TO HYDROGRAPHIC 

POSITIONING SYSTEMS 

If redundant data are available, the least square adjust- 
ment method may be used to compute the coordinates of hydro- 
graphic survey positions. Observation equations may be 
written for various types of survey methods. By expressing 
these equations in matrix notation and using successive approxi- 
mations of the unknowns, the best values for the coordinates 
of survey positions may be determined. The predictable 
accuracy of these best values may also be found. Thus, 
redundant observations, coupled with mathematical data 
adjustment techniques, produce a viable method of system 
calibration for hydrographic survey data. This method of 
calibration is referred to as auto calubration. 

1 . Azimuth Angle Positions 

The working range of azimuthal systems is limited to 
line of sight distances, i.e., 5-15 nautical miles, depending 
upon the height of the observing instrument. 

Because of this range limit, the Universal Transverse 
Mercator (UTM), or other plane coordinate systems, may be used. 

Let 

7l , y 2 » y 3 = Northings of the shore stations 1, 2 and 3, 
respectively, 

x^, x^, x^ = Eastings of the shore stations 1, 2 and 3, 
respectively, 

P = The position of the survey vessel. 
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Then, the azimuth (from north) of the survey vessel from 
shore stations can be written in terms of coordinates as 



Ajp ■» 4arf ' ~^ A j Aj|>=tarr l ^ j A 3 P = tan - ' — 

*?-*i *p-%3 



In these equations, x„. and y are the best estimate of the 

P P 

survey vessel coordinates which are to be determined. 

These equations are non-linear. Thus, in order to 
form observation equations, they must be linearized. 
Letting x p = x 0 + Ax and y p = + ^ where x <> and >"q are 

the approximate coordinates of the vessel's position, and 
using Taylor series expansion for linearization. 



f (^y) Uo>S)o) + ^-A * + J*£_ + , 

where f = A lo ^ -tan’' MgZli . 

The partial derivatives are 

pf _ _ 9o-VK . d£_ _ Ao- Ail 

DA© s <o ^ 

The observation equations now may be written in the follow- 
ing detailed form: 
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— 










y©-y> 






_ 


— - 




S?o 


S<> 








A^p-Avo 


V. 








A* 




Vi 

V 3 


rv 


3©-3a. 


*0-M- 








Aip — A 20 


- P 


*2b 


Soo 








/\3p- A 30 






yo-i)3 






1. 










* 3 © 


*30 







1 



where 

A to 



■boo"' 



5 



and Aip , Aap , A3p 
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are the measured azimuths, and g = 57.2958, the conversion 
factor from radians to degrees. 

Having obtained the observation equation, the normal 
equation may be formed and solved by following the procedures 
outlined in Section II. A. 

Using the computed values of Ax and Ay, new trial 
point coordinates may be formed as follows: 

* — *o , 

J) = • 

The values are substituted in the observation equation for 
the initial x 0 , y Q coordinates. The least square solution 
is iterated until the Ax and Ay values approach zero. 

Example 1 1 1 - 5 : Referring to Figure III -2, the 

coordinates of the shore stations are 



Luces (#1) 
x 4,055,042. 7 m 

y 595,794.5 m 



Mussel (#2) 

4,053,453.2 m 
597,967.8 m 



ME4 (#3) 

4,053,917.2 m 

603,425.2 m 



The standard errors for the azimuth observations are = .02°, 
<?£ = .024° and a 3 = .018°. The following angles were 
measured: 

p - Luces - Mussel = = 50?164 

p - Mussel - Luces = = 99?360 

p - MB 4 - Mussel = a 3 = 47?865 
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Figure III-2: Determination of a position for azimuthal 

systems using the least square method. 
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The least square method will be used to determine the best 
values for the coordinates of the survey vessel. 

Given: A = 126?180 Given: A n = 306?180 

Measured: « 50?164 Measured: a 2 = 99?360 

A lp " 76 ° 016 A 2 p = 4S!S40 

Given: A^ = 265?140 

Measured: = 47?865 

A_ = 313?005 

3P 

First, assume x Q = 4,055,000 

Y 0 = 600,000 



The observation equation, V = AX - L, is then 



p 














" 


V, 




c4 

m 

o 

• 

1 


-,000128 




A* 




-I4.56& 


Va 


— 


-.0\T8S 


.013581- 






— 




V* 




* OIS208 


. 00 480T 




&!) 




25,441 
















- 
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Using the standard errors of each station, a weight matrix 
is formed: 



Vs/ - 



o 



0 

k /« 



2 . 

Oa 



0 



0 

0 






Let k = 0.024, then 



W- 



i *44 
o 
o 



0 



i 

0 



0 

0 

1.18 



Using the A, L and W matrices, the components of the 

T T 

normal equation, A WAX - A WL = 0, are written 



- 










* “ 


. 000991H 


-.0001091 




Ay. 




i.lo32 


.-0001091- 


. 0002.1S1 




Ay 


i 
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The solution, X = (A^WA) ^ (A^WL) , is then 



1059.2 


514. fc 




1,1031 




*12.21.8' 


514.6 






.1131 




1144.0 



Now, the new trial point coordinates are found. 

-4,055,000 -H2.3U8 =■ 4,056 ^ 231 - 8 , 
j = 9o-+^\j -600,000 -tJ144.0 — 6oi, 144^0. 



Using new trial point coordinates, solutions are repeated 
until Ax and Ay vanish. Table I I I - 1 shows the data for other 
trial Ax and Ay values and the new trial point coordinates. 

The best estimate of the coordinates for the survey 
vessel is 

x p = 4,056,302.9 

y = 600,867.4 

P 

2 . Sextant Angle Positions 

In sextant angle positions, similar to the resection 
problem in geodetic work, the measured quantities are the 
included angles at the sounding vessel between the shore 
stations as shown in Figure 1 1 1 - 3 . 
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Iteration Change in Coordinates Coordinates 

No. Ax Ay x 
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Plane coordinates are again used because of the 
visual range limitation. 

It can be seen from Figure III -3 that 

A p2 ' A pl = “l ' 

A p3 ' A p2 = a 2 » 

V ' V = a 3 > 
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or in terms of the coordinates 



too - 1 *»»-«(> _ too ' 1 a * ~ ap - =«(|, 

*l-*p 



t an 



S3- Bp 



W' = o<2. , 

*a.-*p 



too-' ton ' 1 =o<3. 

Xi| — '%p ^3— Kp 



Letting x_ = x + Ax and = y + Ay and linearizing with 

P 0 p o 

Taylor series expansion, 

f^y 1 ) = f(v,y<A-t- Ax + ky + 

"oy-p * 

where 

_ 9^-9o . Sv-Sio 

__ c i 1 

~*2Lo \0 

*f. __ Yu — >>0 Yva— >-o 

"Dy© $\b S20 
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The observation equations may then be expressed as 
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where 

g = 57.2958 is the conversion factor from radians to 
degrees , 

A©i , A© 2 , A^j , A©^ are the computed azimuths of lines 
Ol, 02, 03 , 04 using trial point coordinates x©, y . 

Once establishing the observation equations, the 
solution is found as 



X = (aWl 



The process is repeated until Ax and Ay become very 

small . 

Example 1 1 1 - 6 : Referring to Figure III-4, the 

coordinates of the shore station are 
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Figure III -4: Determination of a position for sextant angle 

fixes using least squares adjustment. 
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MB 4 (#1) 



Use (J 2) 



Mussel (#3) 



Luces (#4) 



x 4,053,917.2 4,051,216.9 4,053,453.2 4,055,042.7 

y 603,425.2 600,372.0 597,967.8 595,794.5 

Measured angles are 
MB4 - p - Use = 49?927 , 

Use-p - Mussel = 38?130, 

Mussel - p - Luces = 30.396. 

The least square method will be used to determine 
the best values for the coordinates of the survey vessel. 

Let the first assumed position be x Q = 4,057,000 and y^ = 599,000. 
Using the first approximate position. 




5 



A q1 = 124?862 , A q2 = 165? 712 , A Q3 = 196?235 and A QA = 238?591 
are obtained. 

The observation equation is 
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and normal equation, A AX - A L = 0, is 
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* DO O l£>Ofc> 


.OOOOO 15 




A* 




-.00 301 


« 0000015 


. 0000 Sll 








* 1511m 



The solution, X = (A^A) * A^L, is 



6129,4 


- 1116 




- .0030 f 




-ki.o 


-in . A 


1 1 401 . 




. 15 W 




031.1 



Then, the new trial point coordinates are 

x = x 0 + Ax = 4,057,000 + (-47) = 4,056,953, 

y = y c + Ay = 599,000 + 1793.1 = 600,793.1. 

Using new trial point coordinates, the above steps are 
repeated until Ax and Ay values become vanishingly small. 

For every trial Ax and Ay value, new trial points coordinates 
are tabulated in Table 1 1 1 - 2 . 

The best values for the coordinates of the sounding 
vessel are 

x p = 4,056,512.3, 

y_ = 600,864.5 . 

P 
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Iteration Change in Coordinates Coordinates 

No. 
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Table 1 1 1 - 2 : For least squares solution, successive iterations 

applied to sextant angle positions 



3. Range-Range Positions 



In range-range positioning, when the distances are 
short (i.e., line of sight type equipment, less than 20 
nautical miles) , a plane coordinate system may be used. Let 
, x^ f = Eastings of the shore stations #1, 2, 3 

Yf, y y^ - Northings of the shore stations #1, 2, 3 
Xp = Easting of the survey vessel 
y^ = Northing of the survey vessel 

Then, the distance between the ith shore station and 
the survey vessel, in plane coordinates, is 



scp = + ( yp - y Ci \ 



This function is non-linear and has to be linearized. 
Introducing approximate coordinates (x^, y Q ) for Xp and 




then 




= AX , 

+ Ay 



Using Taylor series for linearization, the result becomes 



Su P .= -f +2L A* + 2£- A a + 
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After linearization, the observation equation is 
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or in the matrix form, V = AX - L , 
where : 



S 1 , S 2 p, are the measured distances, 

S 10’ S 2o> S 3Q are the computed distances using and y Q . 

In ranging systems, when the distances are long, 
coordinate computations must be carried out on the appro- 
priate ellipsoid using rigorous geodetic formulas. Let 



<t>L > 

A.^ 

iO 





= Computed geographical coordinates, latitude 
and longitude, of the survey vessel, 

= Latitude and longitude of the ith shore station, 
= Azimuth from ith shore station to approximated 
position Oj 

= Azimuth from O to ith shore station, 

= Distance between O and ith shore station. 
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Although the computed observations must utilize rigorous 
geodetic solutions, the differential equations of the observa- 
tions may be approximated using spherical trigonometry [Ref. 3] : 

(is ol = Sin I'* C-£o Cos Aol A ($)© - Rt Cos A Co (A (J) i 
+ N(, Cos Cj) c. St’n Abo ( A\ 0 - AAb) 1 , 

where and are in seconds of arc, dS in meters. Sin 1" 

is the ‘conversion factor from seconds to radians. R^ and R. 

O i 

are the radius of curvature in the plane of meridian at 
point 0 and ith shore station, respectively, defined as 
[Ref. 18] 

R. = o ( >- e?) 

U-e 1 Sm 1 0V >/1 

where a is the semi major axis of the datum ellipsoid, 

2 

and e is the eccentricity of the datum ellipsoid. 

and are the radius of curvature in the plane of prime 
vertical at point O and at ith shore station, respectively, 
defined as [Ref. 18] 

N .= a — 

is the latitude of point O and 0^ is the latitude of the ith shore 
station . 
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The partial derivatives of computed observations 
with respect to parameters are 

* _ Sin I" Ro Cos Aot , 

■5 00 

Ifgh - Sml'Mi. Cos0( $>»o Aio. 

"qXo 

Then, observation equation is 



V, 




— RoCos Aoi , Hi Costal Sirv-Aio 


Va 


= Sinl' 


_ R 0 Cos Aox , Hx CoS^Z S'n_Axo 






— Ro Cos Ao '3 , N3 CoS(J>3 Sin (\%o 
L 



A $ 
AX 



Sp» - So\ 
Spx - Soa 
5p3 - Soi 



where S^, S^ 2 , S ^ are computed distances using inverse 
distance and azimuth formulas (these formulas could be found 
in any geodesy text); and Sp^, Sp 2 , Sp, are measured distances 
between point p and the respective shore station. 

After forming the observation equation, the normal 
equation is found and solved as in previous examples. This 
process is repeated until Acj> and AX become smaller than the 
resolution of the positioning system. 
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Example 1 1 1 - 7 : Referring to Figure 1 1 1 - 5 , the coordi- 

nates of the shore stations are 

Luces ( # 1) Mussel (#2) MB 4 (#3) 

x 4,055,042.7 4,053,453.2 4,053,917.2 

y 595,794.5 597,967.8 603,425.2 

Using the least squares procedure, best values of 
coordinates of the vessel may be found. Let the first 
assumed position x q = 4,056,000 m and y Q = 598,000 m. 

Measured distances are p - LUCES = 4350 m, p - MUSSEL = 4506 m, 
and p - MB4 = 5267 m. For the first approximate position of 
the vessel, the observation equation is written: 
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Normal equation, A AX - A L = 0 , is 
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Figure 1 1 1 - 5 : Determination of a position for range-range 

systems using least squares adjustment. 
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And solution, X = (A T A) 1 A T L, is 
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283*+ ‘.8 
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Ay 




-.019 
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T2>.4 



Then, new trial point coordinates are: 
x = x 0 + Ax = 4,056,000 + 2183.2 = 4,058,183.2 
y * y 0 + Ay = 598, 000 + 787.4 = 598,787.4 

Using the new trial point coordinates, the above steps are 
repeated until Ax and Ay values become smaller than the 
system resolution. For every trial, the change in coordi- 
nates and the coordinates of new trial points are tabulated 
in Table III-3. 

The best values for the coordinates of the sounding 
vessel are 

x„ = 4,057,501.2 

P 

y p = 599,567.7 



The standard errors in the northing and easting may 
also be calculated: 

\J — *"L- ( -for A and V_ j La<>t \ Ve.rc*tCco 'JaLues ace used ^ > 
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Iteration Change in Coordinates Coordinates 
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•— it *3 i 
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~ 2 . 4 ~ 


\TV = 


2 . 1 - 2.6 -i .5 




2.-6 
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- 1-5 



CT 0 





4-. 04 ■ 




T - 1 

where q^'s are the elements of (A A) which has been 
calculated as 




-.14 

.H 



(T* — CTo nT^T -= 4.04- nI • 64~' = ^13 rn , 

0\) cr c xjcj^ - 4.04 'I -16 -3.52m, 



4 . Hyperbolic Positioning Systems 

Hyperbolic positioning systems measure the difference 
in distance from a vessel to the two shore stations. In 
Figure III-6, station number 2 is the master station, and 
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Figure III-6: Determination of a position for hyperbolic 

systems using least squares adjustment 
method. 



110 



numbers 1, 3 and 4 are slaves. Point p is the vessel's 
position, and its coordinates are designated as <pp and Xp. 

Point O is the first approximate position, with and X Q 
representing its coordinates. 

The differential equations of the computed distances 
to each station, S^, may be written as [Ref. 3]: 

ds„(. = Sin I" C-go CosAoL 40 RlCos 

Ml Cos (f) c Sfo. A Co ^ A© — A L ^ "3 

where i represents the shore station number. 

A^ q represents the azimuth from the ith shore station 
to approximate position 0. 

is the radius of curvature in the plane of 
meridian at point O (as defined in Section B.3.). 

is the radius of curvature in the plane prime 
vertical at the ith shore station (as defined in Section B.3.). 
The partial derivatives of the function with respect 

to d> and X^ are 

o o 



- S £L =, _ Sm. 1" Cos AoC . 



= Sfo-^Cos^cSih-ALo. 

Ao 



ill 



The range difference between the distance from the 



vessel to the master and the distance from the vessel to 
the respective slave station is expressed in the equations 
below. 



5fsi _ = Sfn. 1“ Po ( Cos Aoi. - Cos AoO 

~^ s<y2 - _ HC Co 50(1 (C\cl /\io - SCo Au>X 

"ft Xo ^Xo 

Note that station number 2 is the master station, and 
the range difference is stated in terms of the partial 
derivatives . 

With this information, the observation equation is 
written as : 
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where ; 

Aqi , are computed azimuths from approximate 

position 0 to shore station 1, 2 

A 10’ A 20 J *** are com P uted azimuths from shore station 
number 1, 2,... to approximate position 0. 

(S^ - S^) , (Sp 2 - S , ... are measured range 
differences . 

( ‘ S 02 ' s oi^» ( S o2 " S 03- ) ’ •** are com P uted range 
differences . 

Sin 1" is conversion factor from second to radian. 

After writing the observation equation, the normal 
equation is solved and the best estimate of the coordinate 
values is found as previously discussed. 

The process is iterated until A<t> and AX become 
smaller than the standard error of the specific hyperbolic 
system be'ing used. 

5 . Global Positioning System (GPS) 

Global Positioning System fixes are obtained 
utilizing the computed distances from the position of GSP 
satellites to a GPS receiver. The receiver measures the 
arrival of a timing pulse from every satellite within 
acquisition range. The transmit time of each pulse is 
encoded in the received signal. Thus, distance is computed 
using the one way travel time between each satellite and the 
receiver multiplied by the propagation velocity of electro- 
magnetic energy. Three such satellite to receiver ranges 
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may then be applied to solve for the coordinates of the 



receiver. 

Using three satellites to determine a fix results in 
a unique solution for the position coordinates (x, y, z) . 
However, significant error may be induced due to drift in 
the receiver clock. This additional unknown, receiver clock 
bias (E) , may be resolved by processing four satellite 
ranges . 



z coordinate may be input as a known value based on a given 
antenna height above sea level. Thus, the number of unknowns 
will be reduced to three. By using four or more satellites, 
redundant observations are then available so that the data 
can be adjusted by the method of least squares. 

Introducing the following variables, observation 
equations may be written in a straightforward manner: 

R^ , R 2 , Rj , . . . = Measured distances from receiver 

to satellites S^, S 2 , S^, ... 

( x l’ y l » z i) > ( x 2> y 2’ z 2^’ *** = Known positions of 

satellites S^, S^, S^,... 

x, y, z = Unknown position of the observer, p. 

E = Receiver clock bias (unknown) . 

Then, the basic equations are 



For position fixing at sea, it is likely that the 
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^3. - E 4- n/U-y^ + 




Here, the ranges R^ , R^, R^, ... , R n include the actual 
satellite to receiver distance plus some offset due to 
receiver clock error. In the above equations, the 
satellite positions are known, and the four unknowns are the 
user position (x, y, z) and user clock error. 



Taylor series must be applied to form equations suitable for 
use with the method of least squares. Let 



where R. is the distance between a satellite and the user 
position, p. 



Since the observation equations are non-linear, the 



x = x Q + Ax 



z = z c + Az 



y = y o + Ay 



E = E_ + AE . 

o 



Using Taylor series, 



Rip ^ E 0 + yo-yO^-t- Uo-^if + A* 



~ Avi v ■+• AE- 

fclo Rio 
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R. is the distance between a satellite and approxi- 

10 

mated user position. 

Observation equations are written in the following 
detailed matrix form; 
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where 

Rip, R 2 p are the measured distances, 

R 10’ R 20 are com P ute d ranges from the formula 



RU 



— E o 







From this matrix, the normal equation may be formed 
and solved as previously discussed. The process is repeated 
until the values of Ax, Ay, Az and AE approach zero. 
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Example 1 1 1 - 8 : At 0800 Zulu, May 1, 1980, a satellite 

fix was taken using a GPS receiver aboard USNS ACANIA in 
Monterey Bay. The measured distances between the satellites 
and the receiver were 

S pl - 20,640,380.8 m 
Sp2 = 20,357,184.1 m 
S p3 = 23,287,346.8 m 
= 21,699,908.4 m 
S p5 = 25 ,416 ,133.6 m 



The satellite 
S #1 

x 1 = 6,097,294.4 
y 1 = -4,364,543.9 
z 1 = 22,658,876.2 



coordinates were 
S #2 

x 2 = 1,819,274.3 
y 2 = -2,240,846.4 
z 2 = 23,721,192.7 



S #3 

x 3 = 9,268,094.7 
y 3 = 13,290,138.0 
z 3 = 13,622,934.4 



S #4 S #5 

x 4 = -8,198,461.9 x 5 = -21,419,309.7 

y 4 = -18,813,603.1 y 5 = 12,865,351.8 

z 4 = 19,040,626.8 z $ = 4,832,143.1 



Applying the method of least squares to determine the 
user position, first assume: 
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x Q = -2,640,000 
y 0 * -4,235,000 
z 0 = 3,960,000 

= 10,000 . 

o 

For the first iteration, the observation equation 
is written as 
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and the normal equation, A AX - A L = 0 , 
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The 



solution, X = 




T - 1 
(A X A) 



T 

A X L, 



30 40. 9 

' lfc>4 • (> 

-21-os. 2 

-\0S4O.9 



is 



New trial point coordinates are 



: = -2,636,959.1 


z = 3,957,294.8 

o 


■ = -4,236,646.6 


E = -840.9 

o 



Using new trial point coordinates, the above steps 
are repeated until Ax, Ay, Az and AE values become vanish- 
ingly small. For other trials (iterations), the Ax, Ay, Az 
and AE values and new trial points coordinates are tabulated 
in Table III-4. 

The final user coordinates are 
x = -2,636,937.1 
y = -4,236,666.2 
z = 3,957,250.8 . 

And receiver clock bias is E = -869.8 m. 
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Change in Coordinates Coordinates 
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C. USE OF THE ERROR ELLIPSE IN ANALYZING THE ACCURACY 

OF HYDROGRAPHIC POSITIONS 

In the least square adjustment process, the positional 
errors are found in the direction of the x and y (<J> and X) 
coordinate axes. These o x and a values indicate the 
expected displacement of the fix in the direction of the 
coordinate axes, but they do not necessarily define the maxi- 
mum and minimum errors associated with the axes of error 
ellipse (Figure 1 1 1 - 7 ] . 

Maximum and minimum standard errors are found by defining 
the orientation of the error ellipse in terms of the x,y 
coordinate system. Let the coordinate system defining the 
semi-major and semi-minor axes of the error ellipse be u 
and v as indicated in Figure 1 1 1 - 8 . 



The following relationship exists between the ellipse 
(u and v) and the ground (x and y) coordinate system. 



U — Y. Sin 0 +■ Cos 0 
V = * Cos0 -y Sin6 



(III-18) 



In these transformation equations, the angle 9 is the 
rotational angle between the y and u axes (measured clockwise 
from y axis to the u axis) . 

The lengths of the semi-major and semi-minor axes are 
given by : 
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Figure 1 1 1 - 7 : Error ellipses formed at the determined 

positions CD. B. Thomson and D. E. Wells, 
1977 ) . 
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Figure III-8: Error Ellipse. 




ov = 




(HI-19) 



In above equations, a , the standard error o£ unit weight, 
is known from the least square adjustment of point p, q uu and 
qyv are Siven by 1 : 

1 For detailed derivation of these equations see Ref. 16, 
pages 181-183. 
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9vm — ^ + C)%y Si(\ 2.9 (HI-20) 

Sw — TjrlS** "+■ Sy*J ^ + "2 ( 9** ” S^S ^ ^ oS “ 9*3 ^' n > (III-21) 



T - 1 

where q , q and q are the elements of the (A A) 
xx yy xy 

T - 1 

(for unweighted observations) or (A WA) (for weighted 
observations), i.e., 



(A TaT' or Ca t ^kT' 



9^> 


9*y 







Equation 1 1 1 - 2 0 reaches its extreme value, and q uu is 
maximum, when 

tan 26 (III-22) 

9 **~ 933 

Inserting Equation 1 1 1 - 2 2 into III-20 and III-21, and 
defining D as . 

D = C (q„ — (III- 23) 
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q uu and q yv may be written as 



Su* = ± ( q** 4- 9yy -+D ) (III-24) 

9vv ( R*> -t- C|yy — & (II I “ 25) 



and from Equation III-19, the semi-major and semi-minor axes are 



Cl -1 Oo (9»+Ryy (III-26) 

b?=l- cr 0 Hq^ -O'). (III-27) 



Using these expressions, the error ellipse can be con- 
structed at any point whose coordinates were determined by 

T -1 T -1 

least square adjustment if the (A A) or (A WA) matrixes 
are known. 

Example 1 1 1 - 9 : In order to determine the error ellipse 

parameters for the range-range example problem (example III-7), 
T - 1 

recall that (A A) was determined as 

*64 -.14 

-.14 .T-fe 
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4.04. 



and the standard error of unit weight was o q = 

The semi -major and semi-minor axes of the error ellipse, 
according to Equations 1 1 1 - 2 6 and III-27, are found by first 
solving Equation III -23 : 

b — C (.6^ -.T6 ) + 4 (-.I**') J — . 305. 

Then, 

Q — T£ + D ) 

Q a — -1- (4.0^ a (* + +. 305) = \B-9 

Q - 3. ?3> m 

b a 

b* - _L (if.04^ ( .6^ -K?6. - . 3>OS> ^ 9*39 
2 

b = 2.39 

The semi-major axis is 3.73 m, and semi-minor axis is 2.99 m. 
According to Equation III -22, the angle 9 is found: 
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1 (-«lQ 



ton 16 = _ 2 = _ 

26 = 113? 2 

6 = 56?6 which defines the orientation of the error 
ellipse . 

1 . Some Characteristics of the Error Ellipse 

A number of important properties of the error ellipse 

can be obtained by analyzing the equations given in the previous 

section. The existence of the error ellipse points out an 

important fact that the accuracy of the location of a point 

in question is not the same in every direction. An analysis 

of equations 1 1 1 - 26 and 1 1 1 - 2 7 demonstrates that the formulas 

for the semi-major and semi-minor axes are composed of two 

parts: the standard error of unit weight, which defines the 

T - 1 

scale of the error ellipse, and the elements of the (A A) or 
T -1 

(A WA) .matrix, which define its shape. 

To reduce the error ellipse into a circle, where the 
accuracy of position is equal in every direction, the follow- 
ing condition must be met; 

a/b = 1 . 

According to Equations 1 1 1 - 26 and 1 1 1 - 2 7 this is possible 
only if D = 0 : 

D-LCq*»-<ta? + 4<9» 9 f] ,h -o. 
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which means that 



^xx ^yy anc * q xy ’ 
and, according to Equation III -17, 

a = a 
x y * 

Another important characteristic is that the sum of 
the squares of the standard errors in x and y directions is 
invariant to the rotation of the coordinate system, or 

-= CTa ■+■ Cj- •=. CT* + O'J' . (Ill -28) 



Equation III -28 leads to the concept of root mean 
square error, d rms , as follows ; 

d rms = vGF+ff? =\/^I7o? = (III-29) 



or, from Equations 1 1 1 - 2 6 and 1 1 1 - 2 7. 



'rms 



•= cr f 



n/ 0 !** +■ 
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IV. CONCLUSION 



Conventional survey systems will provide the primary means 
of hydrographic positioning for several years to come. 

Thus , the concepts of d rms and the graphical approach to 
developing error contours are very useful tools in survey 
planning and execution. 

Survey planners must exercise care in establishing the 
position of navigation aids. The resultant net geometry 
determines the accuracy, and thus the d rms error, of the fix 
positions. Accuracy requirements for the collection of 
hydrographic survey data greatly limit the size of the 
effective survey area. Through careful planning, the 
number of navigation aid shore stations can be minimized 
while still meeting position accuracy requirements for the 
survey. 

Currently, more research is needed to determine the 
environmental factors which govern variations in the propa- 
gation velocity of electromagnetic energy. If this important 
parameter could be more accurately modeled throughout the 
survey area, the effects of systematic errors due to these 
velocity variations could be greatly minimized. 

As shown in this paper, methods exist today by which the 
accuracy of survey positions can be greatly improved through 
the use of redundant observations and data adjustment techniques. 
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The method of least squares adjustment provides a best esti- 
mate of position, plus the size and orientation of the error 
ellipse associated with that position, for every point deter- 
mined by the survey system. In addition, the error ellipse 
quantifies the predictable accuracy (as shown in Figure II-7) 
of each position as compared to the repeatable accuracy 
available from conventional survey methods. 

The application of these techniques will become more 
widespread when the Global Positioning System is fully 
operational. Observations of position from any number of 
navigation and positioning systems (GPS, LORAN, hydro position- 
ing systems, etc.) can be combined in a least square solution. 
Observation equations may be written and weights can be 
assigned as a function of accuracy for each system. 

In preparation for these future improvements, hydrographers 
must work to understand and implement the concepts discussed 
in this paper. Data must be processed by computer. There- 
fore, programs need to be written which can perform the 
iterative least squares adjustment on the appropriate obser- 
vation equations. Errors must be analyzed to assess the 
improvement in accuracy resulting from redundant observations. 
Additionally, standard hydrographic survey procedures need 
to be reviewed to determine if more efficient methods may 
be adopted when redundant observations and data adjustment 
techniques are used. For example, the method of least 
squares may be programmed into onboard computers so that 
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data may be adjusted and evaluated in real time. Alternately, 
redundant data may be collected and recorded for later pro- 
cessing ashore. 

The technology is available today to employ data adjust- 
ment methods in hydrography. This technology must be 
analyzed and adapted to match the systems and requirements 
unique to hydrographic survey. 
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ANALYSIS OF RANDOM ERRORS 



APPENDIX A 

(The information given in this section was taken directly 
from References 1 and 20.) 

ONE DIMENSIONAL ERRORS 

An error in a measurement is the difference between the 
"true*' value of a quantity and the measured or derived value. 
The "true" value can never really be determined because of 
instrument limitations and human fallibility. In determining 
the value of a quantity, only one measurement may be necessary 
when an approximate value is sufficient. If, on the other 
hand, the quantity is important enough to require a more 
precise value, repeated measurements are made. Variations 
will exist between the values obtained from several measure- 
ments. Applying the theory of the normal distribution to 
these measurements, the "best" value for the quantity is the 
mean or average of all the observed values. The differences 
between the mean and the observed values are the apparent 
errors or residuals which are used to derive a statement of 
precision for the measuring process. When the residuals are 
randomly distributed about the mean, the precision of the 
measurement is expressed by a single term, the standard 
error, which is commonly designated by the Greek letter "sigma" 
(a). For a one dimensional normal distribution, this value 
is computed by squaring all the residual errors (v) , adding 
the squared values, dividing by the number of errors less 
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one (if n independent direct measurements are taken of the 
same quantity, then the first measurement establishes a 
value for the unknown and all additional measurements, 

(n-1) in number, are redundant), and taking the square root: 




where is the residual defined by the equation - x , 

x.^ : observed value 

x ; mean value 

n : the number of observations . 

The normal distribution itself is represented by the 
function : 

p(\j) — — — — e 2 - c ' i . 
r cr^ 

The normal distribution curve and the meaning of the standard 
errors are illustrated in Figure A-l. The central vertical 
axis, p(v), represents the probability of zero error with 
positive errors plotted to the right and negative errors 
to the left. The height of the curve above a particular 
point on the horizontal axis is proportional to the probability 
of an error of that amount. 
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Figure Al: One dimensional Normal Distribution Curve 



It can be observed from the normal distribution curve 
that the total area under the curve is equal to unity. Also, 
the area under the curve between any two values of v^ 
and V 2 equal to the probability of an error occurring 
between these limits. So, to find the probability of an 
error between and v 2 , p(v) has to be integrated between 
and V r The area under the curve between the limits of 
= -a and v 2 = +a is 68.27% of the total area under the 
curve. This means that there is 68.27% probability that 
errors in any further measurements made under the same 
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conditions will not exceed the standard error, a, with a 
68.27% probability. The standard error does not indicate 
the probability that an error of a certain size will occur; it 
only indicates that 68.27% of the errors will fall within 
the specified limits of plus or minus one sigma. 

If other probability levels are desired, the appropriate 
conversion factor may be found in Table Al. For example, 
for 95% probability, a should be multiplied by a linear 
error conversion factor of 2. 



Probability, % 



Linear error 
conversion factor 



50 

68.27 

90 

95 

99.7 



.6745 

1.000 

1.6449 

2.000 

3.000 



Table Al : Linear error conversion factors for several 

probability levels. 

TWO-DIMENSIONAL ERRORS 

A two-dimensional error is the error in a quantity defined 
by two random variables. For example, consider the position 
of a point referred to x and y axes. Each observation of the 

x and y coordinates may contain the errors v and v . 

x y 
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If the errors are random and independent, each error has 
a probability density distribution of 



p (\0 — k e. 2c *’ afiei pWy) — — . e. 2-dy - 

"The probability of two events occurring simultaneously 
is equal to the product of their individual probabilities" 
[Ref. 1], Applying this rule, the two-dimensional probability 
density function becomes: 



PK,V S > = _L £ 1 V ^ 

'i'ft CTvj 

rearranging terms, 

~u.sU -^4 \ 

Itt <t* c\, = e 2 ' ; 




taking the logarithm, 



(-a') JfrvCp cr* 



v* 



X 

Vy 






For given values of p (v v ) [physical meaning of 

x y 

p (v , v ) is that the probability that two random variables 
x y 

v and v take values in the interval ±v and ±v ] , the left 
x y x y 

side of equation is a constant, k then 
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For several values of P( v ^ c > V y) > a family of equal probability 
density ellipses are formed with axes ko^ and kffy. (Figure A2) . 







Figure A2 : Equal probability density ellipses. 

In general, when the two errors are correlated, i.e., 
a change in the one error has some effect upon the other, the 
probability density function, p(v x , v y ) , becomes 
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1 



p (v/*,v* ) = 



.. . . (^1-2 C5*^ 

3^-<r/ y r ff * 



V*V* 



<<y H z 



05 1 



-) 



^ V<yfo a -<* 



•2. 

*y 



Then, the equation of constant probability density ellipses 
(Figure A3) is 



k l =_2 

o-e 1 ) 



v» _ 2p + v ?- 

<y* 0*. ^ 



where p = correlation coefficient of 



v and v 
x y 



and is given by 



^ o+cy 

The probability density function integrated over a certain 

region becomes the probability distribution function which 

yields the probability that v and v will occur simultaneously 

x y 

within that region, or: 



P -Jj p(V%, dv^dv/y 
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Figure A3: Constant probability density ellipse for 

correlated errors. 
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Values of d 



APPENDIX B 

USEFUL GRAPHS FOR THE DETERMINATION OF REPEATABILITY CONTOURS 




Intersection angle, B, in degrees. 
•Figure A1 : For ranging systems, the graph of the d rms /o s 

and e/b. 
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Values of e/b 



rms 




Figure A2 : For azimuthal systems, the graph of d^g/a^b 

and e/b. 
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Values of e/b 
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